FBP: A Frontier-Based Tree-Pruning Algorithm

A frontier-based tree-pruning algorithm (FBP) is proposed. The new method has comparable order of computational complexity, comparing with Cost-Complexity Pruning (CCP). Regarding tree pruning, it provides a full spectrum of information; specifically, (1) given the value of the penalization parameter λ, it gives the minimum size of a decision tree; (2) given the size of a decision tree, it provides the range of the penalization parameter λ, within which the complexity-penalization approach will render such a tree size; (3) it finds the sizes of trees that are inadmissible — no matter what the value of the penalty parameter is, the resulting tree based on a complexity-penalization framework will never have these sizes. Simulations on real datasets reveals a ‘surprise’: in the complexity-penalization approach, most of the tree sizes are inadmissible. FBP facilitates a more faithful implementation of the principle: Cross-Validation (CV). Simulations seem to favor such an approach. Utilizing FBP, a stability analysis for CV is proposed.